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Preface 


The  Integrated  Meteorologieal  System  (IMETS)  is  a  weather  data  system  utilized  by  the  Air 
Foree  weather  foreeasters  in  support  of  Army  operations.  Predietion  and  foreeast  produets  on 
IMETS  are  aehieved  through  the  Battleseale  Foreeast  Model  (BFM)  and  the  Fifth-Generation 
Pennsylvania  State  University/National  Center  for  Atmospherie  Researeh  Mesoseale  Model 
(MM5),  whieh  are  used  for  short-term  and  long-term  foreeasts,  respeetively.  Both  models 
provide  preeipitation  foreeasts  although  different  techniques  have  been  derived  to  provide 
precipitation  output.  This  report  describes  the  precipitation-forecasting  techniques  and  a 
comparison  of  the  BFM  and  MM5  products. 
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INTENTIONALLY  LELT  BLANK. 


Summary 


The  U.S.  Army  Research  Laboratory  has  developed  a  mesoscale  weather  model  called  the 
Battlescale  Forecast  Model  (BFM).  After  model  initialization,  the  BFM  produces  forecast 
variables  for  a  24-h  period.  Since  the  Army  required  a  longer-term  prediction,  the  Mesoscale 
Model  Version  5  (MM5)  gridded  data  are  received  from  the  U.S.  Air  Force  Weather  Agency  to 
provide  forecast  information  for  up  to  a  48-h  period.  Due  to  the  importance  of  precipitation  on 
the  tactical  decision  aids,  as  well  as  military  operations  in  general,  both  models  forecast 
stratiform  and  convective  precipitation,  which  are  made  available  to  the  user  in  a  database  and 
graphically. 

This  report  describes  the  basic  meteorological  theory  applied  to  the  precipitation  processes  and 
forecasts  for  both  the  BFM  and  MM5.  The  effectiveness  of  the  BFM  and  MM5  precipitation 
output  are  analyzed  as  well. 

Precipitation  forecasts  are  derived  from  numerical  model  data,  such  as  the  BFM  and  MM5 . 
These  data  provide  the  forecaster  and  users  with  a  detailed  overview  of  the  atmospheric 
conditions  that  might  produce  precipitation  along  with  the  general  precipitation  rates,  amounts, 
and  types.  These  precipitation  parameters  are  later  placed  into  a  database  so  other  programs, 
such  as  the  Integrated  Weather  Effects  Decision  Aid,  can  attain  this  information. 
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1.  Introduction 


The  Integrated  Meteorologieal  System  (IMETS)  is  a  mobile,  operational,  automated  weather 
data  receiving,  processing,  and  disseminating  system  utilized  by  Air  Force  weather  forecasters  in 
support  of  Army  operations.  The  U.S.  Army  Research  Laboratory  (ARE)  is  supporting  the 
forecaster  to  make  more  specific  and  precise  battlefield  weather  forecasts  by  producing  weather 
products  on  IMETS.  One  product  to  assist  in  short-term  forecasting  (<=24  h)  is  an  operational 
mesoscale  model,  the  Battlescale  Forecast  Model  (BFM).  For  longer-term  data,  the 
Pennsylvania  State  University/National  Center  for  Atmospheric  Research  Mesoscale  Model 
Version  5  (MM5)  output  is  available  from  6  to  48  hours  {1,  2). 

The  BFM  produces  many  forecasting  parameters,  including  temperature,  pressure,  dew  point, 
relative  humidity,  and  windspeed  and  direction  as  well  as  precipitation  amounts.  While  these 
outputs  provide  valuable  weather  information.  Tactical  Decision  Aids  (TDAs)  such  as  the 
Integrated  Weather  Effects  Decisions  Aid  (IWEDA),  have  a  need  for  additional  precipitation 
parameters  such  as  precipitation  rates  and  precipitation  types.  The  IWEDA  has  been  developed 
to  simplify  the  manner  in  which  environmental  impacts  on  weather  systems  are  displayed  to 
users.  The  IWEDA  generates  current  and  forecasted  impacts  on  personnel  and  approximately  70 
weapon  systems,  such  as  attack  helicopters  and  fixed  wing  aircraft.  Both  the  BFM  and  the  MM5 
can  be  used  to  derive  precipitation  amounts  and  rates,  while  a  post-processing  software  package 
has  been  developed  to  forecast  precipitation  types  using  the  model-derived  output  (5). 


2.  Mesoscale  Models  for  the  Army 


The  ARE  implemented  the  Higher  Order  Turbulence  Model  for  Atmospheric  Circulation 
(HOTMAC)  as  their  model  for  the  IMETS  platform  in  response  to  the  Army’s  requirement  for 
small-scale  weather  information  on  the  order  of  less  than  500  by  500  km.  The  HOTMAC  was 
selected  because  it  is  numerically  stable  at  long-time  steps,  globally  relocatable,  emphasizes 
boundary-layer  physics,  and  is  platform-independent.  In  an  effort  to  keep  the  model  run  time  as 
fast  as  possible,  the  BFM  contains  no  convective  cloud  parameterization  or  cloud  microphysics 
package.  The  model  currently  is  run  to  24  h;  however,  due  to  military  requirements,  it  was 
necessary  to  add  the  MM5  to  the  IMETS  platform  to  provide  forecast  grids  out  to  48  h  from  the 
initial  forecast  time  (4,  5). 
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2.1  The  BFM 

The  BFM  contains  16  terrain-following  vertical  levels,  a  model  top  of  7000  m  above  the  highest 
elevation,  a  10-km  horizontal  resolution,  and  a  log-linear  stagger  so  that  there  is  greater  vertical 
resolution  near  the  surface.  The  rapid  run  time  for  the  model  can  be  attributed  to  a  single  nest 
and  no  moist  physics  or  cumulus  parameterization  routines.  However,  because  of  the  implicit 
approach,  time  steps  on  the  order  of  200  s  (at  10-km  resolution)  are  common  for  typical 
atmospheric  advective  speeds  and  vertical  motion  fields  in  the  model.  Soil  temperature  on  five 
subsurface  levels  is  solved  using  the  heat  conduction  equation,  while  long  and  shortwave 
radiation  within  a  single  layer  for  a  stratus  cloud  is  calculated  using  the  method  of  Hanson  and 
Derr.  The  basic  variables  that  are  prognostically  forecasted  by  the  model  are  perturbation 
potential  temperature,  total  water  substance  mixing  ratio,  wind  speed,  wind  direction,  pressure, 
soil  temperature,  turbulence  kinetic  energy  and  length  scale,  and  non-convective  precipitation 
rate  {6,7). 

To  initialize  the  BFM,  surface  data  and  upper-air  observations  are  input  into  the  model  in  the 
area  of  interest.  Additionally,  the  36-h  forecasted  Naval  Operational  Global  Atmospheric 
Prediction  System  (NOGAPS)  package,  which  is  issued  by  the  Air  Force  Weather  Agency 
(AFWA)  via  the  Air  Force  Automated  Weather  Distribution  System,  is  utilized  as  the  long-range 
data  that  the  BFM  is  nudged  toward.  The  NOGAPS  grid  points  are  spaced  1“  apart,  both  latitude 
and  longitude,  on  the  mandatory  pressure  surfaces.  Lateral  and  time-dependent  boundary 
conditions  (large-scale  forcing)  are  supplied  from  grid-point  data  close  to  the  area  of  interest 
taken  from  NOGAPS  output  valid  at  analysis  and  forecast  times  of  interest. 

The  BFM-generated  outputs  for  the  grid  include  the  u  and  v  horizontal  wind  vector  components, 
potential  temperature,  and  water  vapor  mixing  ratio.  These  forecast  fields  are  saved  at  0,  3,  6,  9, 
12,  15,  18,  21,  and  24  h  from  the  base  time  of  the  model  run  and  placed  into  a  Gridded 
Meteorological  Data  Base  (GMDB). 

2.2  The  MM5 

The  MM5  is  a  limited-area,  non-hydrostatic,  terrain-following  sigma-coordinate  model  designed 
to  simulate  or  predict  mesoscale  and  regional-scale  atmospheric  circulation. 

Terrestrial  and  isobaric  meteorological  data  are  horizontally  interpolated  from  a  latitude- 
longitude  mesh  to  a  variable  high-resolution  domain  on  Mercator,  Lambert  Conformal,  or  polar 
stereographic  projection.  Since  the  interpolation  does  not  provide  mesoscale  detail,  these 
interpolated  data  may  be  enhanced  with  observations  from  the  standard  network  of  surface  and 
rawinsonde  stations  using  either  a  Cressman  or  multiquadric  scheme.  In  the  MM5,  there  is  also 
a  program  that  performs  the  vertical  interpolation  from  pressure  levels  to  sigma  coordinates. 

The  sigma  surfaces  near  the  ground  closely  follow  the  terrain,  while  the  higher-level  sigma 
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surfaces  tend  to  approximate  isobarie  surfaces.  Additionally,  the  MM5  has  a  flexible  and 
multiple  nesting  capability,  advanced  physical  parameterization,  3-D  data  assimilation  system 
via  nudging,  and  it  can  be  run  on  various  platforms  (5). 

Version  3  of  the  MM5  was  used  for  this  study;  it  is  from  AFWA  and  has  a  resolution  of  15  km 
mesh  data  on  41  vertical  levels.  The  ARL  reeeives  these  MM5  data  in  gridded  binary  form  for 
the  Continental  United  States  twice  each  day,  which  are  initialized  at  0600  universal  time 
coordinated  (UTC)  and  1800  UTC,  respeetively.  Due  to  computational  and  processing 
constraints,  there  is  a  6-h  stagger  between  the  initialization  valid  time  of  the  15-km  mesh  and  the 
first  forecast  output,  thus  the  first  MM5  forecast  is  a  6-h  forecast.  The  frequency  of  the  model 
output  is  every  3  h,  for  a  time  period  of  48  h. 

The  current  AFWA  operational  version  of  MM5  places  the  lowest  model  vertical  level  at  20 
magi.  To  generate  data  at  the  standard  observation  heights  of  10  magi  and  2  magi,  similarity 
theory  is  being  used  at  ARL  to  extrapolate  to  these  lower  levels  from  the  lowest  MM5  sigma 
level.  In  this  fashion,  temperature,  dew  point,  and  wind  data  at  levels  2  magi  and  10  magi  are 
produced  at  ARL  in  addition  to  the  41  MM5  sigma  levels  of  data. 

The  parameterizations  selected  by  AFWA  with  this  version  of  the  MM5  are  as  follows: 

•  Grell  cumulus  parameterization  -  Designed  for  grid  sizes  of  10  to  30  km,  this 
parameterization  aceounts  for  subgridscale  eonveetion  and  compensating  subsidence. 

•  MRF  planetary  boundary-layer  model  -  Parameterizes  the  mixture  of  heat,  moisture, 
and  momentum  in  the  boundary  layer. 

•  Reisner  mixed  phase  explicit  moisture  microphysics  -  Cloud  and  rainwater  fields  and 
ice  proeesses  are  predicted  explieitly.  No  graupel  or  riming  proeesses  are  ealculated. 

•  Dudhia  cloud  radiation  -  Provides  solar  and  infrared  fluxes  at  the  ground  and 
atmospheric  tendencies  resulting  from  the  radiative  proeesses. 

•  MM5  five-layer  soil  model  -  Temperature  predicted  in  1,  2,  4,  8,  16  cm  layers  with  fixed 
substrate  below  using  vertical  diffusion  equation. 


3.  Cloud  Condensation  and  Non-Convective  Precipitation  from  the  BFM 


The  relative  humidity  of  each  of  the  16  layers  produeed  by  the  BFM  is  one  key  parameter  needed 
to  caleulate  the  preeipitation  amount  and  precipitation  rates  in  the  BFM.  Sinee  the  BFM  does 
not  directly  calculate  the  relative  humidity;  it  was  neeessary  to  derive  this  value  using  eq  1 : 
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RH  = 


TWTR 


SWVAP 


where 

RH  =  relative  humidity  for  eaeh  grid  point 
TWTR  =  total  water  at  each  grid  point 
SWVAP  =  saturation  vapor  pressure  for  the  grid  point 
These  calculations  are  done  for  each  of  the  16  levels  at  each  BFM  grid  point. 

3.1  Total  Liquid  Water  and  Cloud  Fractions 

In  the  BFM,  the  mixing  ratio  of  total  water  is  a  combination  of  the  mixing  ratio  of  water  vapor 
and  the  mixing  ratio  of  cloud  water  as  noted  in  eq  2: 

Q  =2+0 

^  w  ^  V  C  I 


where 

Qw  =  mixing  ratio  of  total  water 
Qv  =  mixing  ratio  of  water  vapor 
Qc  =  mixing  ratio  of  cloud  water 

The  value  of  Qv,  the  mixing  ratio  of  the  water  vapor,  is  derived  from  the  initial  data  of  the  model. 
In  model  operation,  Qc  is  initially  0. 

Recalling  that  the  change  of  total  water  with  time  is  conserved,  the  approach  suggested  by 
Sommeria  and  Deardoff  is  used  to  derive  the  value  of  Qc,  the  mixing  ratio  of  cloud  water  (P). 

The  liquid  water  potential  temperature  is  defined  as: 

o.  =  e-^^Q, 

where 

6i  =  liquid  water  potential  temperature 
0  =  potential  temperature 
T  =  absolute  temperature 
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Lv  =  latent  heat  of  eondensation 

Cp  =  speeifie  heat  of  dry  air  at  constant  pressure 

In  order  to  recover  the  potential  (or  absolute)  temperature  and  mixing  ratios  of  water  vapor  (Qv), 
the  probability  density  function,  G,  defined  by  Sommeria  and  Deardoff  (P),  is  used.  The  density 
function  is  assumed  to  be  Gaussian  such  that: 


G= 


exrt 


where 

0/  =  fluctuation  of  liquid  water  potential  temperature 
qw  =  fluctuation  of  total  water  mixing  ratio 


and: 


+ 


2  ^ 


'^^qw  J 


(4) 


(5) 


(J 


2 

qw 


(6) 


^  91^  qw 

The  local  condensation  is  given  by: 

Qi={Q.-QsYh{x) 

where 

Qs  =  the  saturation  mixing  ratio, 

H(x)  =  Heaviside  function, 

X  ~  Qw'Qs 


(V) 


(8) 
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defined  as: 


H(x)  =  0,  x<0 
H(x)=l,  x>0 

The  final  equation  for  eloud  water  mixing  ratio  is  expressed  as: 


2cr, 


RQ,+ 


V2 


exp 


71 


f 


V 


a 


2  V 


) 


(9) 


(10) 


where 

R  =  eloud  fraetion 


(11) 


where 

'^Q  =Qw-Qs/ 

u]  ^\l\a^ql -2abq„e,  +b^0^)  ds. 

The  values  of  a  and  b  in  eq  12  are  eonstants  as  defined  in  Yamada  (10)  while  the  value  of  R  in  eq 
10  is  a  funetion  of  eloud  eoverage  for  a  given  volume  of  air  as  given  by  Mellor  (11). 

Figure  1  shows  how  the  eloud  fraetion  R  varies  as  a  funetion  of  Q/,  a  statistical  ratio  of  super  or 
subsaturation.  In  the  figure,  clouds  can  exist  even  if  the  mixing  ratio  of  water  vapor  over  a  grid 
is  not  saturated.  This  is  realistic  since  the  grid  spacing  normally  used  in  mesoscale  models  is 
larger  than  the  size  of  small  clouds. 
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Figure  1.  Relationship  Showing  the  Cloud  Fraction  as  a  Statistical  Function. 

However,  after  investigating  this  method  for  cloud  fractions  it  was  determined  that  it  was  not  an 
effective  way  to  calculate  clouds.  Some  of  the  reasons  for  this  decision  included: 

•  The  value  of  cloud  liquid  water  is  often  too  low  due  to  the  coarse  resolution  in  both  the 
horizontal  and  vertical  in  the  BFM. 

•  Saturation  is  not  well  represented  in  the  sounding  data  used  for  initialization  of  the  BFM. 

•  The  BFM  nudges  to  forecasted  NOGAPS  data,  which  contains  its  own  biases. 

•  Original  design  of  the  BFM  was  for  stratiform  precipitation  in  the  lowest  4  km. 

•  The  assumption  that  the  grid  spacing  is  larger  than  the  size  of  the  smaller  clouds  may  be 
accurate;  however,  stratiform  clouds  are  often  in  layers  and  are  larger. 

•  The  Sommeria  and  Deardorff  (P)  method  to  derive  cloud  fractions  was  not  designed  for 
mid-levels  or  for  multiple-layer  cases. 

•  The  technique  developed  by  Sommeria  and  Deardorff  (9)  was  designed  for  much  smaller 
grid  volumes  than  the  current  BFM. 

Ironically,  the  statistical  cloud  model  was  a  method  formulated  to  avoid  coarse  grid  models  in 
which  the  saturation  values  are  lowered  arbitrarily  to  compensate  for  the  cloud  that  was  not 
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resolved;  however,  the  technique  of  training  or  “learning”  of  the  data  did  provide  much  more 
realistic  and  accurate  cloud  fractions  in  the  BFM.  While  it  may  appear  that  small  differences  in 
relative  humidities  are  not  important,  Walcek  proved  that  a  2  to  3  percent  increase  in  relative 
humidity  could  lead  to  a  15  percent  increase  in  cover.  Additionally,  Shultz  and  Politovich 
observed  that  relative  humidity  values  in  excess  of  55  percent  between  500  to  1000  mbar  usually 
identify  regions  with  widespread  cloudiness  on  the  Nested  Grid  Model  {12, 13). 

The  BFM  also  had  this  bias,  with  clouds  observed  in  layers  well  below  saturation.  To 
compensate  for  this,  Passner  developed  a  routine  based  on  a  set  of  IF-THEN  rules  with  an 
emphasis  on  season,  time  of  day,  location,  and  layer  relative  humidity.  This  cloud  program  was 
used  on  both  the  BFM  and  MM5  with  statistical  evaluation  showing  the  software  to  be  most 
effective  in  the  lowest  4000  ft  of  the  atmosphere  (14). 

The  importance  of  the  cloud  fraction  is  seen  in  the  next  section  when  discussing  the  precipitation 
rates  produced  by  the  BFM. 


3,2  Precipitation  Rates  from  the  BFM 

Since  the  microphysical  processes  of  stratiform  precipitation  are  not  part  of  the  BFM,  the 
stratiform  precipitation  is  parameterized  as  a  function  of  cloud  liquid  water.  The  scheme 
formulated  by  Sundqvist  et  al.  for  stratiform  precipitation  is  used  in  the  BFM  (15). 


The  rate  of  release  of  precipitation  is  described  by: 


P  =  Ga 


r 


1-exp 


r 


V 


g 

\P*Qc,crJ 


2^ 


j 


(13) 


where 

Co  =  characteristic  time  for  the  conversion  of  cloud  droplets  into  raindrops 
Qc  =  mixing  ratio  of  cloud  water  content 
R  =  cloud  fraction 

Qc.cr  =  Cloud  water  content,  at  which  release  of  precipitation  starts  to  be  efficient 

According  to  Sundqvist  {15),  Qc,cr  should  have  a  value  typical  of  individual  cloud  types  and  be 
invariant  to  grid  resolution.  He  also  suggests  a  value  of  0.0001  for  Co,  which  equates  to  a 
conversion  time  of  approximately  167  min.  In  his  study,  Sundqvist  worked  with  the  operational 
fine  mesh  model  of  the  Norwegian  Meteorological  Institute,  which  uses  a  horizontal  grid 
resolution  of  50  km.  In  this  test,  for  the  BFM,  a  value  of  0.0004  (42-mm  conversion  rate)  was 
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employed  for  the  model  runs.  This  value,  Co,  was  found  to  be  one  of  the  more  influential  values 
in  determining  the  accumulated  precipitation. 

The  rate  of  precipitation  P,  at  a  given  z*  level,  is  given  by: 


(h  +  z 


g  max 


V 


H 


p(z*)Pdz 


(14) 


where 

z*  =  vertical  coordinate  system  used  in  the  BFM 
H  =  depth  of  the  model  atmosphere 
Zgmax  =  highest  terrain  elevation  in  the  BFM  domain 
Zg  =  terrain  elevation 
P  =  air  density 

To  simulate  the  coalescence  process,  Sundqvist  et  al  (15)  introduced  an  additional  parameter, 
which  increases  with  the  rate  of  precipitation.  Additionally,  when  the  temperature  is  lower  than 
-5  °C,  to  simulate  an  enhanced  release  of  precipitation  in  clouds  containing  a  mixture  of  droplets 
and  ice  crystals  (Bergeron-Findeisen  mechanism),  he  adds  another  parameter.  Finally,  since 
cirrus  clouds  contain  mainly  ice  crystal  growth  by  diffusion,  Sundqvist  uses  another  parameter 
that  increases  with  decreasing  temperature. 

In  the  stratiform  case,  evaporation  of  precipitating  water  is  assumed  to  take  place  according  to 
the  relation: 


CF  (15) 

where 

Fe  =  is  a  coefficient  in  the  expression  for  evaporation  of  precipitation  (0.00001  in  this  case) 
U  =  relative  humidity  of  the  layer 
CF  =  cloud  fraction  for  the  grid  volume 
V  Pr  =  precipitation  rate 

The  final  form  of  the  stratiform  precipitation  rate  at  the  surface  is: 
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dz 


(16) 


where 

CoF  =  Parameter  to  include  Bergeron-Findeisen  mechanism  for  precipitation  formation 
Pw  =  density  of  water 

The  subscript  F  in  the  term  Q(c,cr)F  indicates  that  the  additional  parameter  for  the  coalescence  and 
Bergeron-Findeisen  mechanism  are  included.  The  final  precipitation  rate  is  expressed  in 
millimeter  per  hour. 


4.  Precipitation  Rate  from  the  MM5 


The  MM5  has  many  different  ways  to  treat  precipitation  physics.  The  explicit  schemes  treat 
resolved  precipitation  physics  while  implicit  schemes  treat  the  non-resolved  precipitation 
physics.  In  the  MM5  version  being  discussed  here,  the  explicit  scheme  is  used  with  the  Reisner 
mixed-phase  ice  scheme.  The  scheme  is  activated  whenever  grid-scale  saturation  is  reached. 
The  equations  for  water  vapor,  cloud  water  (ice),  and  rain  water  (snow)  mixing  ratios  are  based 
on  the  conservation  of  moisture  but  add  the  effects  of  the  Reisner  microphysics  package.  An 
example  of  these  equations,  the  equation  for  rain  water  (snow  if  below  0  °C)  mixing  ratio,  is: 


dt 


^ uq  ^ vq 


m 


m 


dc 


^q  <j  „  ^Frfgq 


where 

m  =  map  factor 
p*  =  p  star 

qr  =  mixing  ratio  of  cloud  water 
a  =  sigma 

5nh  =  non-hydrostatic  constant 
DIV  =  divergence 
Vf  =  fall  speed  of  rain  or  snow 
p  =  density  of  air 
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G  =  acceleration  of  gravity 

Pre  =  the  evaporation  of  rain  and  sublimination/deposition  of  snow 
Prc  =  eonversion  of  cloud  to  rain  (ice  to  snow) 

Pra  =  accretion  of  eloud  by  rain  (ice  by  snow) 

Dqc  =  diffusion  term 
PsM  =  snow  melt 

Pci  =  heterogeneous  freezing  of  eloud  water  to  cloud  ice 

The  terms  Psm  and  Pci  are  the  two  terms  added  to  the  simple  iee  phase  scheme.  In  the  Reisner 
seheme,  snow  does  not  melt  instantaneously  above  0  °C.  Additionally,  supercooled  water  ean 
exist  below  0  °C  and  unmelted  snow  ean  exist  above  0  °C.  Separate  arrays  are  used  to  store 
vapor,  eloud,  eloud  ice,  and  snow. 

The  mixing  ratio  of  rain  water  is  used  as  a  key  parameter  in  the  fall  speed  term,  whieh 
determines  the  rainfall  rate  at  the  surfaee.  The  equation  for  the  fall  speed  is: 

r(4  +  b) 

=  a - 

6  <>«> 


where 

Vf  =  fall  speed 
f  =  gamma  function 
a  =  84 1 .9946  for  rain  or  1 1 .72  for  snow 
b  =  0.8  for  rain  or  0.41  for  snow 
The  value  of  k  from  eq  18  is  determined  in  eq  19: 

/  N  1/4 

I  p^r  y 

where 

71  =3.1416 

No  =  Marshall-Palmer  intereept  parameter  8x10^  m'"^ 
p  =  mean  air  density  of  rain  or  snow  particles  (1000  and  100  kg  m‘  ) 


(19) 
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5.  Statistical  Evaluation  of  Mesoscale  Models  and  Precipitation 


Two  types  of  evaluations  are  done  in  this  study.  The  first  will  be  for  “YES/NO”  foreeasts,  where 
the  foreeast  provides  information  if  a  eertain  weather  phenomena  will  or  will  not  oeeur.  A 
seeond  type  of  evaluation  is  where  the  model  output  is  investigated  for  “error”  or  how  much  the 
predicted  value  differs  from  the  observed  value. 

5,1.  Evaluation  of  “YES/NO”  Forecasts 

A  contingency  table  provides  a  statistical  method  to  display  answers  to  binary  YES/NO 
forecasts.  Some  evaluation  techniques  include  the  probability  of  detection  (POD),  false  alarm 
rate  (EAR),  the  correct  non-event  (CNE),  critical  success  index  (CSI),  true  skill  score  (TSS),  and 
bias.  The  calculations  are  based  on  the  contingency  elements  listed  in  table  1,  while  the 
equations  for  the  evaluation  techniques  are  also  shown. 


Table  1.  Contingency  Table  for  Eorecasted 
and  Observed  Weather  Event. 


Forecast 

YES 

Forecast 

NO 

Observed  YES 

A 

B 

Observed  NO 

C 

D 

POD 

FAR 

CNE 


A 


A 

-b 

B 

c 

c 

+ 

A 

D 

D 

+ 

c 

(20) 


(21) 

(22) 


Donaldson  developed  the  CSI,  which  considers  three  of  the  four  elements  in  the  contingency 
table;  however,  it  does  not  take  into  account  the  D  element  (null  element).  Hanseen  and  Kuipers 
formulated  an  equation  that  does  factor  in  the  null  event,  and  called  it  the  TSS  {16, 17). 


CSI 


A 

A  +  B  +  C 


(23) 
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TSS 


{AD  )  -  (BC  ) 
{A  +  B){C  +  D) 


(24) 


The  bias  in  a  forecast  is  the  ratio  of  the  number  of  positive  forecasts  to  the  number  of  observed 
events,  as  shown  in  eq  25: 

A  + 

A  +  B  (25) 


5,2  Error  Evaluation 

The  three  main  products  used  in  this  study  to  evaluate  model  or  post-processed  derived  output 
are  mean  absolute  difference  (AD),  root-mean  square  error  (RMSE),  and  correlation  coefficient 
(CC).  The  equations  are: 


m  n 


m*  n  (26) 

where 

X  =  meteorological  variable 
O  =  observation 
P  =  prediction  of  variable 
i  =  i*  surface  station 
j  =  j*  forecast  day 
n  =  number  of  stations, 
m  =  total  number  of  forecast  days 

Small  values  of  AD  are  related  to  good  agreements  between  observation  and  forecast. 
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RMSE 

The  values  of  RMSE  are  proportional  to  those  of  the  AD.  The  CC  is  displayed  in  eq  28.  The  CC 
measures  the  strength  of  the  relationship  between  two  variables.  When  CC  >0,  it  indieates  a 
positive  linear  relationship.  A  value  of  1.00  indieates  a  “perfeet”  eorrelation  between  the 
observed  and  predieted  values  of  a  meteorologieal  foreeast. 


m  n 


6,  Evaluation  of  Precipitation  Forecasts 

There  were  approximately  25  model  runs  done  in  a  variety  of  loeations  in  the  United  States; 
however,  there  was  an  emphasis  on  typieal  wintertime  eases  and  stratiform  preeipitation  sinee 
the  main  goal  was  to  study  preeipitation  rates,  preeipitation  type,  and  the  resulting  surfaee 
visibility. 

To  verify  these  data,  hourly  surfaee  observations  were  randomly  used  at  a  variety  of  unique 
terrain  loeations  on  the  grid.  This  was  done  so  that  the  influenee  of  terrain  eould  be  ineluded  on 
the  resulting  preeipitation  totals.  Eaeh  hourly  surfaee  observation  ineludes  a  eoded  value  for  the 
aeeumulated  preeipitation  over  the  past  hour.  Unfortunately,  the  preeipitation  rates  produeed  by 
the  BEM  are  instantaneous  rates,  sueh  as  0.06  in/hr,  while  the  MM5  rates  are  an  average  rate 
determined  by  the  total  preeipitation  output  from  the  model  over  a  3-h  period.  Assumptions 
must  be  made  that  the  model  preeipitation  is  a  steady  rate,  whieh  may  be  a  safe  assumption  for 
stratiform  preeipitation,  although  stratiform  preeipitation  ean  vary  with  time.  An  effort  was 
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made  to  eliminate  all  eonveetive  preeipitation  eases  in  this  study.  This  is  not  always  feasible  as 
even  the  most  uniform  storms  sometimes  eontain  mesoseale  features  that  ean  enhanee 
preeipitation  on  smaller  seales. 


6,1  Results  of  the  BFM  and  MM5  Precipitation  Forecasts 

The  most  basie  evaluation  of  the  model  preeipitation  forecasts  was  to  investigate  how  well  the 
model  forecasted  precipitation  at  any  hour  of  the  model  runs.  Table  2  displays  these  data. 


Table  2.  “YES/NO”  Forecasts  of  Precipitation 
During  the  Winter  Season  in  2003. 


Model  Precip 

BFM 

MM5 

Samples 

501 

463 

POD 

0.66 

0.82 

FAR 

0.35 

0.42 

ONE 

0.79 

0.70 

CSI 

0.48 

0.51 

TSS 

0.45 

0.52 

Bias 

1.02 

1.42 

Cases  with  precip 

37% 

33% 

The  results  indicate  that  the  MM5  has  a  higher  POD  of  forecasting  precipitation,  although  it  does 
have  a  slightly  higher  FAR  and  is  biased  toward  overforecasting  precipitation.  The  BFM  data 
set  does  have  a  slightly  higher  percentage  of  cases  with  precipitation,  although  this  difference  is 
not  significant  enough  to  bias  the  results. 

6,2  Precipitation  Rates 

It  is  impossible  to  derive  the  instantaneous  precipitation  from  a  surface  observation;  therefore, 
the  precipitation  rates,  as  already  mentioned,  are  not  exactly  matched.  However,  these  data  in 
tables  3  and  4  do  give  the  user  a  valuable  glimpse  of  rainfall  intensity  from  the  models. 
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Table  3.  Statistical  Analysis  of  Precipitation  Rates  from 
the  BFM. 


BFM 

hours 

Samples 

RMSE 

(mm/h) 

CC 

Fcst  Ave 
(mm/h) 

Observed 

Ave 

(mm/h) 

00 

31 

3.87 

0.12 

0.38 

1.81 

3 

22 

1.43 

0.15 

0.56 

1.01 

6 

26 

2.36 

0.42 

0.63 

1.18 

9 

22 

1.40 

0.09 

0.65 

1.13 

12 

17 

1.74 

-0.15 

0.67 

1.05 

>12 

30 

1.28 

-0.05 

0.61 

0.85 

Totals 

148 

2.01 

0.10 

0.58 

1.17 

Table  4.  Statistical  Analysis  of  Precipitation  Rates  from 
the  MM5. 


MM5 

(hours) 

Samples 

RMSE 

(mm/h) 

CC 

Fcst 

Ave 

(mm/h) 

Observed 

Ave 

(mm/h) 

09 

17 

3.50 

0.02 

0.40 

1.39 

12 

22 

2.80 

0.21 

0.83 

1.50 

15 

19 

1.25 

-0.13 

0.35 

0.78 

18 

19 

2.86 

0.20 

1.46 

1.22 

21 

16 

1.57 

0.33 

1.24 

0.70 

>=24 

22 

0.94 

0.19 

0.70 

0.41 

Total 

115 

2.15 

0.14 

0.83 

1.00 

Tables  3  and  4  show  the  results  of  the  precipitation  rates.  As  expected,  the  CC  are  very  low 
because  of  the  wide  disparity  in  rainfall  prediction.  This  is  also  seen  in  the  RMSE  column, 
where  both  models  show  a  high  error.  Overall,  the  sample  size  for  the  hourly  data  is  rather 
small;  however,  there  are  interesting  trends  noted  in  these  data.  For  both  the  BFM  and  MM5,  the 
initial  time  period  shows  the  lowest  forecasted  precipitation  rates.  According  to  Dudhia,  the 
precipitation  may  take  several  model  time  steps  between  production  and  when  it  finally  reaches 
the  ground.  In  the  BFM,  this  may  be  a  function  of  slow  moistening  of  the  atmospheric  column 
and  lower  mixing  ratio,  cloud  fraction,  precipitation  rate,  along  with  a  high  evaporation  rate  (18). 

After  the  initial  forecast  period,  the  BFM  precipitation  rates  are  nearly  constant  through  the  24-h 
model  run.  Additionally,  the  observed  precipitation  shows  little  variation  through  the  data.  This 
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is  not  the  case  with  the  MM5  output,  which  shows  more  fluctuation  in  both  the  forecast  averages 
and  observed  averages.  Of  great  interest  is  the  trend  in  the  MM5,  where  the  precipitation  rates 
are  less  than  the  observed  precipitation  rates  through  the  flrst  15  h  of  the  model  runs  and  then 
suddenly  changes  at  the  18-h  period  when  the  forecasted  precipitation  becomes  greater  than  the 
observed  rates.  The  signiflcance  of  this  trend  and  its  cause  is  uncertain  because  of  the  small  data 
sample. 

6,3  Precipitation  Type 

An  interesting  question  is:  Does  the  precipitation  type  have  any  influence  in  the  rainfall  rates, 
snowfall  rates  or  total  amounts?  In  this  study,  the  routine  developed  at  ARL  is  used  to  determine 
if  the  precipitation  will  reach  the  surface  as  rain,  snow,  freezing  rain,  or  some  mixture  of  rain  and 
snow.  The  routine  is  implicit,  so  it  is  run  as  part  of  the  post-processor  from  the  BFM  and  MM5. 
Using  this  method,  only  the  lowest  10,000-ft  above  ground  level  (AGL)  is  used,  since  most 
stratiform  precipitation  falls  from  clouds  below  that  level  and  the  temperature  is  almost  always 
below  0  °C  in  typical  wintertime  precipitation  above  that  level.  Listed  below  are  some  of  the 
key  assumptions  of  the  precipitation-type  software: 

•  Uses  the  forecasted  wet  bulb  temperatures  rather  than  temperature. 

•  Goes  vertically  from  surface  and  counts  layers  above  and  below  0  °C. 

•  If  all  layers  are  below  freezing,  then  precipitation  will  be  snow.  If  all  layers  are  above  0  °C 
then  precipitation  will  be  rain  at  the  surface. 

•  Freezing  rain  is  forecasted  when  some  layer  above  the  surface  is  above  0  °C  and  the 
surface  is  at  0  °C  or  less. 

•  Calculates  the  depth  of  the  elevated  warm  layer,  which  will  help  determine  if  falling  snow 
will  melt. 

•  Calculates  the  near  surface-layer  average  temperature  to  know  the  depth  of  any  warm  or 
cold  layers  near  the  surface. 

•  Does  checks  to  see  if  snow  will  melt  before  reaching  ground  or  rain  will  freeze  at  the 
surface. 

•  If  the  routine  finds  a  borderline  case  between  rain  and  snow,  it  becomes  a  “mixed”  case. 

During  the  winter  season  of  2003,  nearly  500  surface  observations  were  collected  to  coincide 
with  areas  where  the  BFM  and  MM5  were  run.  The  emphasis  in  the  BFM  was  for  all  forecasts 
less  than  12  h  and  for  the  MM5  from  9  to  24  h.  In  table  5,  the  results  of  the  precipitation  type 
study  from  the  BFM  are  shown,  while  table  6  shows  similar  results  from  the  MM5 . 
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Table  5.  BFM  Precipitation-Type  Forecasts  (Horizontal) 

and  Observations  (Vertical)  for  all  Forecast  Hours 
499  Samples). 


Fcst/Obs 

None 

Rain 

Snow 

Freezing 

Rain 

Mixed 

None 

249 

36 

21 

0 

0 

Rain 

49 

51 

5 

0 

0 

Snow 

13 

14 

38 

0 

1 

Freezing 

Rain 

8 

5 

2 

0 

1 

Mixed 

1 

6 

0 

1 

0 

Table  6.  MM5  Precipitation-Type  Forecasts  (Horizontal) 
and  Observations  (Vertical)  for  all  Forecast  Hours 
(461  Samples). 


Fcst/Obs 

None 

Rain 

Snow 

Freezing 

Rain 

Mixed 

None 

218 

67 

15 

0 

3 

Rain 

20 

55 

1 

0 

0 

Snow 

13 

8 

44 

0 

3 

Freezing  Rain 

1 

6 

0 

1 

0 

Mixed 

1 

4 

0 

0 

1 

Tables  5  and  6  show  encouraging  results,  especially  in  the  snow  forecasts.  In  84  percent  of  the 
BFM  snow  cases,  snow  was  forecasted,  while  98  percent  of  the  snow  forecasts  were  correctly 
predicted  in  the  MM5.  There  was  a  higher  error  in  the  rain  forecasts,  although  the  POD  of  rain 
was  still  67  percent  in  the  BFM  and  75  percent  in  the  MM5.  However,  it  should  be  noted,  the 
error  of  forecasting  rain  and  having  freezing  rain  occur  is  a  function  of  the  models  not 
forecasting  the  surface  temperature  cold  enough.  Even  an  error  of  0.1  °C  can  cause  this  forecast 
to  be  incorrect.  As  noted  by  Passner  {14),  the  BFM  tends  to  overforecast  the  surface  temperature 
when  the  boundary  layer  is  moist,  thus  it  is  not  surprising  to  see  1 8  percent  of  the  snow  cases 
being  forecasted  as  rain  cases  due  to  this  high  temperature  bias.  The  MM5  has  a  slight  bias  to 
underforecast  the  temperature  in  moist  environments,  thus  this  cold  bias  helps  to  drive  the  MM5 
surface  temperature  lower  and  results  in  a  very  high  POD  for  snow  forecasting.  The  main  bias  in 
the  precipitation-type  software  is  that  too  many  rain  forecasts  are  actually  being  observed  as 
snow,  freezing  rain,  or  mixed  precipitation. 
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The  sample  size  for  freezing  rain  and  mixed  precipitation  was  very  small  as  only  about  4  percent 
of  all  the  precipitation  observations  were  freezing  rain  and  approximately  4  percent  were  mixed 
precipitation.  As  noted  in  the  tables,  the  precipitation-type  software  rarely  forecasts  freezing 
rain  or  mixed  precipitation,  most  likely  because  the  models  cannot  achieve  a  detailed  enough 
profile  of  the  temperature  and  moisture. 

A  final  area  to  investigate  was  how  the  precipitation  rates  varied  with  the  precipitation  type  in 
each  model.  Table  7  shows  the  differences  in  the  forecasted  and  observed  precipitation  rates  for 
rain  and  snow  with  the  BFM  and  MM5. 


Table  7.  Precipitation  Rates  and  Precipitation  Types  for  the 
BFM  and  MM5  (All  Hours). 


Model  and 
Precip  Type 

Samples 

RMSE 

(mm/h 

) 

Forecast 

Avre  (mm/h) 

Observed 

Ave(mm/h 

) 

BFM  Snow 

34 

0.80 

0.54 

0.75 

BFM  Rain 

93 

1.32 

0.68 

1.54 

MM5  Snow 

31 

0.84 

0.37 

0.58 

MM5  Rain 

63 

2.70 

1.04 

1.31 

The  results  in  table  7  show  that  precipitation  rates  and  observed  rates  are  significantly  lower  for 
snow  than  for  rain.  Both  models  underforecast  the  snowfall  rates,  with  the  BFM 
underforecasting  snowfall  rates  by  28  percent  while  the  MM5  underforecast  the  snowfall  rates 
by  36  percent.  The  BFM  does  have  a  more  significant  error  in  rainfall  rates  with  an  error  of  56 
percent  in  the  rates  while  the  MM5  rainfall  rates  are  underforeacsted  by  21  percent.  The  rainfall 
rates  are  higher  than  snowfall  rates  because  there  is  more  available  liquid  water  in  the 
atmosphere  and  the  mixing  ratio  values  are  higher. 


7.  Summary  and  Discussion 


This  study  was  designed  to  investigate  the  precipitation  rates,  precipitation  amounts,  and 
precipitation  types  forecasted  from  two  mesoscale  models,  the  BFM  and  MM5.  A  description  of 
how  precipitation  is  formulated  in  each  model  helps  to  enhance  the  understanding  of  how  these 
factors  influence  the  model  output.  The  BFM  stratiform  precipitation  rates  depend  on  the 
precipitation  rate  itself  and  the  evaporation  of  the  precipitation  as  it  falls  through  the  atmospheric 
layers.  The  most  significant  terms  for  the  precipitation  rate  is  the  cloud  liquid  water  of  the  layer, 
the  cloud  fraction  of  the  layer,  the  coefficient  for  cloud  water  content  at  which  the  release  of 
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precipitation  becomes  efficient,  and  the  coefficient  for  the  conversion  rate  of  cloud  particles  to 
precipitation  size.  The  evaporation  rate  in  the  BFM  depends  on  the  relative  humidity  of  the 
layer,  the  cloud  fraction,  the  precipitation  rate  for  the  column,  and  a  constant  derived  by 
Sundqvist  (15). 

In  the  MM5,  the  stratiform  precipitation  routine  is  an  explicit  scheme,  where  the  scheme  is 
activated  when  grid-scale  saturation  is  reached.  There  is  an  explicit  treatment  of  cloud  water, 
rain  water,  snow,  and  ice  along  with  feedback  to  the  temperature  and  moisture  field  along  with 
the  radiation  scheme.  The  MM5  contains  a  microphysics  package  known  as  the  mixed-phase 
Reisner  microphysics  package,  which  builds  upon  the  simple  ice  routine  by  permitting 
supercooled  water  below  0  °C  and  has  a  gradual  snow  melt  as  it  falls.  Additionally,  unmelted 
snow  can  exist  above  0  °C.  The  value  of  the  mixing  ratio  is  used  in  the  final  fall  term  in  the 
MM5 .  This  fall  term  is  the  actual  precipitation  that  reaches  the  ground. 

The  statistical  evaluation  of  the  models  provided  many  useful  hints  on  how  to  improve  and 
upgrade  the  model.  The  BFM  is  underforecasting  precipitation  rate  by  nearly  50  percent,  while 
the  MM5  is  underforecasting  precipitation  rates  by  17  percent  for  the  overall  model  sample.  The 
MM5  has  an  interesting  trend,  where  the  model  underforecasts  rates  by  43  percent  through  15  h 
and  then  overforecasts  the  rates  by  156  percent  from  15  to  48  h  after  model  initiation.  Both 
models  produce  lower  precipitation  rates  in  snow  than  rain,  and  it  was  found  that  the  models 
rarely  produce  snowfall  rates  (liquid  equivalent)  greater  than  1.00  mm/h.  The  BFM  error  is 
logical,  given  the  model’s  dry  bias  and  the  problems  with  excessive  evaporation  below  cloud 
base;  however,  the  trends  in  MM5  precipitation  rates  are  more  complex  since  it  contains  a 
microphysics  package  with  many  assumptions  about  cloud  nuclei  sizes,  density,  and  nuclei 
amounts. 

The  most  vital  role  of  the  precipitation  rates  is  that  they  influence  the  prevailing  surface  visibility 
in  the  post-processing  software.  Knapp  developed  regression  equations  based  on  2790  surface 
observations  using  two  types  of  equations;  one  with  a  known  ceiling  but  no  precipitation  falling 
and  another  with  a  ceiling  along  with  precipitation.  Passner  noted  that  model  biases  were 
influencing  visibility  forecasts  and  that  the  equations  Knapp  formulated  were  not  working  well 
with  the  BFM  and  MM5  output.  To  compensate  for  these  results,  rainfall  and  snowfall  rates 
were  used  to  help  determine  precipitation.  As  an  example,  when  snowfall  rates  of  1.75  to  2.54 
mm/hr  were  produced  by  the  model,  the  forecasted  visibility  was  one  mile  {19, 14). 

Table  8  shows  the  performance  of  the  models  under  different  precipitation  conditions,  which  are 
the  result  of  a  visibility  test. 
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Table  8.  BFM  and  MM5  Visibility  Errors  Based  on  Observed 
Winter  Weather  in  2003. 


Model/ 

Obstruction 

Forecast 
Ave  (miles) 

Observed 
Ave  (miles) 

Mean 

AD 

Samples 

BFM  No 
Precipitation 

7.68 

9.67 

2.18 

151 

MM5  No 
Precipitation 

8.14 

9.67 

1.68 

198 

BFM  Fog 

5.50 

3.50 

4.00 

62 

MM5  Fog 

5.68 

3.68 

3.30 

50 

BFM  Rain 

5.76 

4.80 

3.01 

112 

MM5  Rain 

5.32 

4.90 

3.10 

83 

BFM  Snow 

5.49 

1.97 

3.90 

63 

MM5  Snow 

6.08 

2.45 

4.46 

72 

The  results  in  Table  8  show  the  model  visibility  foreeasts  are  aeeurate  when  no  preeipitation  is 
falling.  When  fog,  rain,  or  snow  is  observed,  the  models  overforeeast  visibility  in  all  three  eases. 
The  fog  oases  are  using  the  original  visibility  equations  from  Knapp  {19)',  however,  the  rain  and 
snow  oases  are  based  on  the  adjustments  made  for  preeipitation  rates.  The  mean  AD  is  generally 
the  same  in  all  three  oases;  however,  the  most  signifioant  error  appears  to  be  with  the  snow 
oases,  whioh  are  overforeoasted  on  average  by  3.6  miles. 

In  74  peroent  of  the  snow  oases,  the  observed  surfaoe  visibility  was  less  than  2  miles;  however, 
the  average  snowfall  foreoast  in  this  study  was  5.75  miles.  A  future  step  will  be  to  lower  the 
snowfall  rates  and  the  foreoasted  visibilities  sinoe  the  models  are  not  able  to  physioally  produoe 
the  preeipitation  intensity  that  is  often  observed.  The  other  major  ohange  in  the  BFM  will  be  to 
use  the  70-minute  oonversion  rate,  whioh  should  enhanoe  the  rainfall  and  snowfall  rates.  These 
two  teohniques  should  make  a  dramatio  improvement  in  the  post-prooessed  visibility  routine. 
Additional  testing  will  be  oonduoted  to  evaluate  how  these  ohanges  work  with  an  independent 
data  set  in  a  variety  of  winter  oonditions.  Additional  evaluation  of  preeipitation  foreoasts  must 
also  be  oompleted  with  small-soale  models  suoh  as  a  5-km  MM5  to  see  if  the  foreoast  are 
sensitive  to  grid  resolution. 
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Acronyms 


AD 

absolute  difference 

AFWA 

Air  Force  Weather  Agency 

AGL 

above  ground  level 

ARL 

Army  Research  Eaboratory 

BFM 

Battlescale  Forecast  Model 

CC 

correlation  coefficient 

CNF 

correct  non-event 

CSI 

critical  success  index 

FAR 

false  alarm  rate 

GMDB 

Gridded  Meteorological  Database 

GriB 

gridded  binary  form 

HOTMAC 

Higher  Order  Turbulence  Model  for  Atmospheric  Circulations 

IWEDA 

Integrated  Weather  Effects  Decision  Aids 

IMETS 

Integrated  Meteorological  System 

mbar 

millibar 

MM5 

Mesoscale  Model  Version  5 

NOGAPS 

Naval  Operational  Global  Atmospheric  Prediction  System 

POD 

probability  of  detection 

RMSE 

root-mean  square  error 

TDAs 

Tactical  Decision  Aids 

TSS 

true  skill  score 

UTC 

universal  time  coordinated 
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